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SUMMARY 


We  derive  the  structure  of  the  solution  of 


f(x)  ^Ma|^g(y)^h(x-y)+f(ay-»-b(x-y))  J 

% 

In  the  case  where  g  and  h  are  concave  is 
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NOTES  IN  THE  THEORY  OP  DYNAMIC  PROGRAMMING  -  II: 

A  FUNCTIONAL  EQUATION  ARISING  IN  ALLOCATION  THEORY 

by 

Richard  Bellman 

$1 •  Introduction 

An  equation  we  have  used  for  illustrative  purposes  in  a 
number  of  papers  is 

(1)  f(x)  -  Max  C  g(y)+h(x-y)+f  (ay+b (x-y) )  3 

0<y<x 

This  equation  arises  in  connection  with  a  multi-stage  allocation 
process  which  we  shall  describe  below. 

The  purpose  of  the  present  note  is  to  complete  the  descrip¬ 
tion  of  the  solution  we  have  given  elsewhere,  [l] ,  for  the  case 
where  g  and  h  are  concave,  monotone  increasing  functions.  The 
method  of  successive  approximations  that  we  employ  is  a  very 
powerful  analytic  tool  for  the  treatment  of  functional  equations 
of  this  general  class. 

§2.  Description  of  the  Process 

We  are  given  a  resource,  x,  to  divide  into  two  parts,  y  and 
x-y.  Prom  y  we  obtain  a  return  of  g(y);  from  x-y  a  return  of 
h(x-y).  In  so  doing  we  expend  a  certain  amount  of  the  original 
quantity  and  are  left  with  a  new  quantity  ay-fb(x-y),  where 
0  <  a,  b  <  1.  This  process  is  now  continued  indefinitely.  How 
does  one  allocate  at  each  stage  so  as  to  maximize  the  overall 
return  from  the  process? 
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If  we  let  f(x)  denote  the  overall  return  obtained  employing 
an  optimal  policy.  It  la  easy  to  ate  that  an  application  of  the 
principle  of  optimality,  of.  [2],  [3],  yields  the  functional  equa 
tlon  In  (l.l). 

§3-  The  Character  of  the  Solution 
The  result  we  wish  to  prove  la 
Theorem .  Let  us  assume  that 

(1)  a.  g(x)  and  h(x)  are  both  strictly  concave  for  x  £  0, 
monotone  Increasing,  and  g(0)  ■  h (0 )  *  0. 

b.  g'(0)/(l-a)  >  h'  (0 )/ ( 1 — b )  #  h'(0)  >  g'(oo),  b  >  a. 


Remark:  We  have  given  the  result  only  for  one  set  of  the 

possible  Inequalities  connecting  g * (0 )  and  h'(0).  It  is  easy 
to  see  that  the  same  methods  suffice  to  cover  the  other  cases. 


Proof:  Let  us  proceed  by  successive  approximations.  For  the 

results  we  shall  tacitly  assume  concerning  the  existence  and 
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unlqueness  of  the  solution  of  (l.l)  and  the  convergence  of  the 
successive  approximants,  we  refer  to  [l] ,  and  [3].  The  criterion 
above  in  (3)  for  the  determination  of  y(x)  is  constructive  since 
we  can  always  obtain  f(x)  directly  by  means  of  an  iterative  pro¬ 
cess.  Here,  however,  we  are  primarily  interested  in  the  structure 
of  the  solution. 

Set 


(4)  fi(x)  «  Max  [g(y)  +  h(x-y)  ]  . 

0<y<x 

Since,  by  assumption,  g'(0)  >  h'(0),  for  small  x  we  have 
g'(y)  -  h'(x-y)  >  0  for  0  <  y  <  x.  Hence  g(y)  +  h(x-y)  is 
monotone  Increasing  in  0  <  y  <  x  and  the  maximum  occurs  at 
y  -  x.  As  x  increases,  the  equation  g’(y)  -  h'(x-y)  -  0  will 
ultimately  have  a  root  at  y  ■  x,  and  then  a  root  inside  the 
interval  [o,x].  The  critical  value  of  x  is  given  as  the  solu¬ 
tion  of  g'(x)  -  h' (0)  ■  0.  This  equation  has  precisely  one 
solution,  which  we  call  x».  For  x  £  l®t  yi  •  y 1 (x )  be  the 
unique  solution  of  g'(y)  -  h’(x-y).  The  uniqueness  is  a  con¬ 
sequence  of  the  concavity  assumptions  concerning  g  and  h. 

Thus  we  have 

(5)  fi (x)  -  g(x)  ,  0  <  x  <  xi  , 

-  g(yi)  +  h(x-yi)  ,  x  >  xt  • 


and 
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(6)  f  i(x)  -  g’ (x)  ,  0  <  x  <  x i 

-  C  g'  (yi)-h'  (x-y»)  +  h'(x-yi)  -  h'(x-yj), 

for  x  >  X|. 

Since  yi(xi)  ■  x»,  Me  see  that  fl(x)  is  continuous  at  x  -  xi,  and 
hence,  for  all  values  of  x  £  0. 

Now  let  us  turn  to  the  second  approximation 

(7)  f  a(x)  -  Max  [  g{y)+h(x-y )+f  i  (ay+b(x-y ) )  ]  . 

°<y<x 

The  critical  function  is  now  D(y)  ■  r'  (y)-h’  (x-y)+f  1 (ay+b(x-y) ) (&-b) . 
Since  g' (0)— h'  (0)+f  1(0)  (a— b)  -  g^Oj-h'  0)+g' (0) (a-b)  > 
h*(0)[  (l-a)  (l+a-b)/(l-b)-l  ]  >  0,  we  see  that  D(y)  is  again 

positive  for  all  y  in  [0,x]  for  small  x.  Hence  the  maximum  occurs 

In  (7)  at  y  -  x  for  small  x.  As  x  increases,  there  will  be  a  first 
value  of  x  where  D(x)  -  0.  This  value,  x«,  is  determined  by  the 
equation  g*(x)  -  h'  (0)+(b-a)f  l(ax) .  Comparing  the  two  equations 

(8)  g'(x)  -  h* (0) 

g'(x)  -  h'(0)  ♦  (b-a)fi(ax)  , 

we  see  that  0  <  x*  <  X|. 

Henoe  the  equation  for  x«  has  the  simple  form 

(9)  g'(x)  -  h'(0)  +  (b-a)g'  (ax)  . 

Thus  y  •  x  for  0  <  x  £  x9  in  (7)  and  y  -  y*(x)  for  x  £  x«, 

where  ya(x)  is  the  unique  solution  of 
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(10)  g'(y)  -  h’(x-y)  +  ( b-a  )f  1  (ay+b (x-y ) )  . 

Furthermore 

(11)  fi(x)  -  g’(x)  ,  0  <  x  <  xa 

-  h'  (x-y*)  +  b  f  i(ayt+b(x-yi))  ,  x  >  x,  , 

i  .  . 

and  f 2 (x )  la  continuous  at  x  -  x*. 

Comparing  (10)  with  the  equation  g 1  (y )  -  h'(x— y)  defining 
y»,  we  see  that  y«(x)  <  yi(x).  In  order  to  carry  out  the  Induc¬ 
tion  and  obtain  the  corresponding  results  for  all  members  of  the 
sequence  £fnJ.  »  defined  recurrently  by  the  relation 

•  Max  [  g(y)+h(x-y)+f  (ay+b(x-y) )  ],  we  require  the  essen- 
n+A  0<y<x  n 

tlal  Inequality  fa(x)  £  fl(x).  There  are  three  Intervals  [0,x«] , 
[x*,xi],  [xj.oo] ,  to  examine,  each  one  requiring  a  separate 
argument.  Using  (10)  and  (ll)  we  have 

(12)  ,  bg' (y2)  -  ah' (x-y2) 

f«(x)  - 

b-a 

for  x  >  x2.  Combining  (6)  and  the  equation  for  yt  we  have 

(13)  .  %  bg'(yj)  -  ah' (x-yt) 

f»(x)  - - 

b-a 

The  function  [  bg’ (y)-ah' (x-y)  ]/(b-a)  is  monotone  decreasing 
in  y  for  0  <  y  <  x.  Since  y*  <  yi  we  see  that  f*(x)  >  f»(x).  This 
completes  the  proof  for  the  Interval  [xi,a>].  The  Interval  [0,Xa] 
yields  equality.  The  remaining  interval  is  [xa,xi].  In  this 
Interval,  we  have 
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(14)  fi(x)  -  g*(x) 

fi(x)  -  W . 

Hence  in  this  interval,  sinoe  Q  <  y*  <  x, 

(15)  ,  bg'(x)  -  ah* (0) 

f«(x)  2 -  2  g'  (x)  , 

b  -  a 

since  g'(x)  2  b'(0)  is  a  consequence  of  g'(y)  2  b*  (x-y)  for 
0  <  y  <  x  and  0  <  x  <  X|.  This  completes  the  proof  that 
f«(x)  2  f i(x). 

We  now  have  all  the  Ingredients  of  an  inductive  proof  which 
shows  that 

(16)  a.  X|  >  Xf  >  •••  gjj  ^  •••  >  0 

b.  f  1  (x )  <  f a (x)  <  f^(x)  <  ... 

c.  y»(x)  >  y*(x)  >  ... 

Since  fR(x)  converges  to  f(x),  f^(x)  bo  f'(x),  yn(x)  to  y(x)  and 
xR  to  x,  we  see  that  the  solution  has  the  indicated  form. 
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